Papers with fine-grained Proximal Policy Optimization
Aligning Large Language Models via Fine-grained Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | Pre-trained large-scale language models often generate biased or toxic text, misaligning with human intentions. |
| Approach: | They propose to use human feedback to improve LLM alignment by fine-grained token supervision . they ask annotators to edit less preferred responses to make them more favorable . |
| Outcome: | The proposed method improves LLM alignment by up to 5.1% in terms of win rate compared with the traditional model. |